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Abstract 

Current  production  navigation  systems  for  agricultural  vehicles  rely  on  GPS  as  the  primary  sensor  for  steering  control.  In  citrus 
groves,  where  the  tree  canopy  frequently  blocks  the  satellite  signals  to  the  GPS  receiver,  an  alternative  method  is  required.  This 
paper  discusses  the  development  of  an  autonomous  guidance  system  for  use  in  a  citrus  grove.  The  vehicle  used  for  this  study  was 
a  common  tractor.  Machine  vision  and  laser  radar  (ladar)  were  individually  used  for  guidance  and  a  rotary  encoder  was  used  to 
provide  feedback  on  the  steering  angle.  A  PID  controller  was  developed  to  minimize  the  path  error.  The  vehicle’s  guidance  accuracy 
was  tested  in  flexible  test  paths  constructed  of  common  hay  bales.  Path  tracking  performance  was  observed.  The  guidance  system 
guided  the  tractor  automatically  through  straight  and  curved  paths.  An  average  error  of  2.8  cm  using  machine  vision  guidance  and 
an  average  error  of  2.5  cm  using  ladar  guidance  was  observed,  when  the  vehicle  was  tested  in  a  curved  path  at  a  speed  of  3.1  m/s. 
The  guidance  system  successfully  guided  the  vehicle  in  a  citrus  grove  alleyway. 

©  2006  Elsevier  B.V.  All  rights  reserved. 
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1.  Introduction 

Florida  supplies  over  80%  of  United  States’  supply  of  citrus.  Over  the  years,  the  labor  supply  for  citrus  harvesting 
has  declined.  Immigration  restrictions  have  limited  the  movement  of  migrant  workers.  The  citrus  industry  is  also  facing 
increased  competition  from  overseas  markets.  As  a  result,  the  need  for  automation  is  being  felt  in  the  citrus  industry. 
An  important  part  of  the  automation  process  is  vehicle  guidance.  Sample  autonomous  vehicle  applications  may  include 
harvesting,  disease  or  nutrient  deficiency  monitoring,  mowing,  spraying  and  other  tasks. 

There  are  numerous  autonomous  and  tele-operated  vehicle  systems  described  in  the  literature.  Tele-operation  has 
been  used  for  guiding  an  HST  drive  vehicle  by  Murakami  et  al.  (2004).  The  major  challenges  encountered  in  tele¬ 
operation  are  the  time  delay  in  communication  and  the  full  time  attention  required  of  a  human.  Yekutieli  and  Pegna 
(2002)  used  a  sensing  arm  to  sense  plants  in  the  path  for  guidance  in  a  vineyard.  However,  using  an  arm  would  require 
citrus  groves  to  be  even  with  continuous  canopy.  There  are  additional  concerns  about  damaging  the  tree  branches. 
Ultrasonic  sensors  have  been  used  for  guidance  in  greenhouses,  but  they  require  that  the  target  be  perpendicular  to 
the  sensor  for  the  ultrasonic  waves  to  be  reflected  back  properly  (Subramanian  et  al.,  2004).  Dead  reckoning  is  widely 
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used  in  combination  with  other  sensors  for  autonomous  vehicles.  Nagasaka  et  al.  (2004)  and  Kodagoda  et  al.  (2002) 
have  used  rotary  encoders  for  detecting  vehicle  position.  However,  wheel  slip  can  cause  significant  error  in  distance 
measurements.  GPS,  in  combination  with  inertial  navigation  systems,  are  used  in  other  areas  of  agricultural  vehicle 
guidance  and  precision  agriculture.  Both  real-time  kinematic  (RTK)  GPS  and  differential  GPS  (DGPS)  allow  accurate 
real-time  measurement.  Nagasaka  et  al.  (2004),  Benson  et  al.  (2001)  and  Noguchi  et  al.  (2002)  have  found  that  RTK 
GPS  receivers  give  very  accurate  results.  Ehsani  et  al.  (2003)  evaluated  the  dynamic  accuracy  of  some  low  cost  GPS 
receivers  with  the  position  information  from  RTK  GPS  as  reference.  They  found  that  these  receivers  had  an  average 
absolute  cross-track  error  around  1  m,  when  traveling  in  a  straight  line.  GPS  cannot  be  effectively  used  for  positioning 
in  citrus  applications,  since  the  vehicle  frequently  moves  under  tree  canopy,  which  blocks  the  satellite  signals  to  the 
GPS  receiver.  Moreover,  a  system  using  GPS  for  guidance  requires  that  a  predetermined  path  be  given  for  the  vehicle 
to  follow.  Consequently,  significant  time  has  to  be  spent  in  mapping  its  path. 

Laser  radar  (ladar)  has  been  used  for  ranging  and  obstacle  avoidance.  Carmer  and  Peterson  (1996)  discussed  the  use 
of  ladar  for  various  applications  in  robotics.  Autonomous  vehicle  navigation  was  discussed  as  a  promising  application 
for  ladar,  due  to  its  ability  to  accurately  measure  position.  Gordon  and  Holmes  (1988)  developed  a  custom  built  ladar 
system  to  continuously  monitor  a  moving  vehicle’s  position.  The  testing  was  done  indoors.  Ahamed  et  al.  (2004)  used 
a  ladar  for  developing  a  positioning  method  using  reflectors  for  infield  road  navigation.  They  tested  differently  shaped 
reflectors  to  determine  the  accuracy  in  positioning.  Ladar  has  been  used  for  navigating  a  small  vehicle  through  an  orchard 
(Tsubota  et  al.,  2004).  A  guidance  system  using  ladar  was  found  to  be  more  stable  than  using  a  GPS  in  a  citrus  orchard 
setting.  In  the  research  mentioned  above,  ladar  was  tested  outdoors  in  the  absence  of  high  dust  conditions.  It  is  expected 
that  the  operation  of  ladar  in  dust,  fog  and  similar  conditions  which  block  light,  would  result  in  reduced  performance. 

Numerous  researchers  have  applied  machine  vision  to  mobile  robots.  Its  low  cost  and  good  performance  have 
made  it  a  good  candidate  for  the  main  guidance  sensor.  Machine  vision  and  ladar  guidance  system  performance  can 
be  comparable  (Subramanian  et  al.,  2004).  Misao  (2001)  used  machine  vision  for  an  automatic  steering  system  on 
a  field  sprayer.  A  video  camera  was  used  to  acquire  the  image  of  the  travel  path  with  red  targets.  Image  processing 
algorithms  determined  the  distance  from  the  current  vehicle  position  to  the  target,  and  then  the  actual  position  to  the 
desired  vehicle  path  was  compared  and  corrected  by  automatic  steering  control.  Han  et  al.  (2004)  developed  a  row 
segmentation  algorithm  based  on  &-means  clustering  to  segment  crop  rows.  This  information  was  then  used  to  steer  a 
tractor.  The  guided  tractor  was  able  to  perform  field  cultivation  in  both  straight  and  curved  rows.  Okamoto  et  al.  (2002) 
developed  an  automatic  guidance  system  for  a  weeding  cultivator.  A  color  CCD  camera  acquired  the  crop  row  images, 
and  by  processing  the  images  in  the  computer,  determined  the  offset  between  the  machine  and  the  target  crop  row.  The 
weeding  machine  was  then  steered  through  the  crop  row  using  an  electro-hydraulic  steering  controller. 

A  good  control  system  is  necessary  irrespective  of  the  guidance  sensor.  Leedforward  +  PID  control  worked  well  for 
guiding  a  tractor  through  crop  rows  (Kodagoda  et  al.,  2002).  PID  control  has  been  used  for  guiding  a  grain  harvesting 
tractor  (Benson  et  al.,  2003).  In  that  research,  PID  was  used  to  calculate  the  actuator  command  signal  based  on  the 
heading  offset.  The  performance  of  the  controller  was  comparable  to  that  of  manual  steering.  Cho  and  Ki  (1999)  used 
fuzzy  logic  controller  and  machine  vision  for  guiding  an  autonomous  sprayer  vehicle  through  orchards.  The  input 
information  to  the  fuzzy  logic  controller  was  given  by  both  machine  vision  and  ultrasonic  sensors. 

To  date,  there  has  been  minimal  success  in  developing  commercial  autonomous  navigation  systems  for  citrus  grove 
applications.  However,  other  similar  applications  have  provided  valuable  insights  for  this  research.  The  vehicle  used  in 
this  research  was  a  John  Deere  6410  tractor.  Machine  vision  and  laser  radar  were  used  as  the  primary  guidance  sensors. 
PID  control  was  selected  for  controlling  the  steering  of  the  autonomous  vehicle,  because  it  has  performed  very  well  in 
previous  mobile  robotics  research. 

2.  Objectives 

The  overall  objective  is  to  design  guidance  systems  based  on  machine  vision  and  ladar  that  will  make  the  tractor 
capable  of  navigating  through  an  alleyway  of  a  citrus  grove.  Before  testing  in  a  citrus  grove,  it  was  decided  to  test  the 
vehicle  in  a  portable  path  constructed  of  hay  bales.  The  following  specific  objectives  were  selected  for  this  research: 

1.  Develop  electro-hydraulic  steering  circuit  for  the  vehicle. 

2.  Develop  a  mathematical  model  for  the  guidance  dynamics  of  the  vehicle  and  design  a  PID  control  for  steering 

control. 
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3.  Develop  two  algorithms  for  path  finding,  one  using  machine  vision  and  the  other  using  laser  radar. 

4.  Test  the  guidance  system’s  performance  on  a  test  track  and  confirm  guidance  in  a  citrus  grove. 

3.  Materials  and  methods 

A  John  Deere  6410  tractor  (Moline,  Illinois)  was  selected  for  this  study.  It  is  well  suited  for  citrus  environments  and 
is  commonly  used  in  spray  applications.  The  width  of  the  tractor  was  approximately  2.5  m.  To  accommodate  electronic 
steering,  the  tractor  was  retrofitted  with  a  Sauer  Danfoss  PVG  32  proportional  servo  valve  (Ames,  Iowa).  The  valve 
was  plumbed  in  parallel  with  the  existing  hydrostatic  steering  system,  as  shown  in  Fig.  1.  The  valve  required  an  analog 
voltage  signal  between  0  and  10  V  as  input. 

A  Stegmann  incremental  rotary  encoder  (Dayton,  Ohio)  was  mounted  on  the  front  steering  cylinder  for  wheel  angle 
feedback.  A  common  2.4  GHz  P4  processor  running  Windows  2000  operating  system  was  used  for  processing  the 
vision  and  ladar  algorithms.  A  “Tern  586  engine”  microcontroller  (Davis,  California)  was  used  for  executing  real-time 
control  of  the  servo  valve  and  encoder  feedback  loop.  An  amplifier  circuit  was  used  to  scale  the  control  voltage  from 
the  microcontroller  to  the  requirements  of  the  valve.  The  guidance  system  architecture  is  shown  in  Fig.  2. 

The  encoder  was  calibrated  by  turning  the  wheels  to  different  angles  and  recording  the  number  of  pulses  given  by 
the  encoder.  Calibration  was  determined  as  7  pulses/0  of  turn. 

Power  was  taken  from  the  tractor’s  12  V  dc  battery  and  inverted  to  120  V  ac  for  running  the  PC  and  the  ladar.  The 
12  V  dc  tractor  supply  was  used  to  power  the  valve  and  the  microcontroller. 

The  camera  used  for  machine  vision  was  a  Sony  (NY,  USA)  FCB-EX780S  “block”  single  CCD  analog  color  video 
camera  with  a  standard  lens  and  resolution  of  640  x  480.  A  frame-grabber  board  was  used  to  convert  the  analog  signals 
to  digital  images.  The  ladar  used  was  the  SICK  LMS  200  (Minneapolis,  Minnesota),  which  has  a  maximum  sweep  angle 
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Fig.  1.  Electro-hydraulic  retrofit  of  the  tractor  for  automatic  guidance. 


Fig.  2.  Guidance  system  architecture  of  the  vehicle. 
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Fig.  3.  Camera  and  ladar  mounted  on  top  of  the  tractor  cab. 


of  180°  and  a  maximum  range  of  80  m.  RS  232  serial  communication  at  1 15,200  baud  was  used  to  communicate  with 
the  ladar.  A  Starfire  (John  Deere,  IL,USA)  Differential  DGPS  receiver  with  30  cm  accuracy  was  used  for  determining 
the  vehicle  dynamics  under  open  field  conditions.  The  camera,  ladar  and  DGPS  receiver  were  mounted  on  top  of  the 
cab.  The  camera  and  the  ladar  were  mounted  at  a  45°  angle  looking  1.5  m  in  front  of  the  tractor  as  shown  in  Fig.  3. 

For  machine  vision,  color  was  used  as  the  discriminator  for  segmenting  the  path.  To  account  for  the  varying  weather 
conditions,  images  were  collected  over  a  period  of  6  days  in  2  months  from  morning  to  evening  at  half  an  hour  intervals. 
Based  on  this  database  of  images,  a  segmentation  algorithm  was  developed  as  shown  in  Fig.  4.  From  the  set  of  images, 


Fig.  4.  Machine  vision  algorithm  flowchart. 
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three  types  of  conditions  were  observed  that  require  the  threshold  value  to  be  changed.  These  are  cloudy  days  when  the 
trees  are  darker  than  the  path;  bright  sunny  days  when  the  trees  are  darker  than  the  path  but  all  pixel  intensity  values 
are  elevated;  and  early  morning  and  evening,  when  the  sunlight  causes  the  trees  on  one  side  of  the  row  to  be  brighter 
than  the  path  and  the  trees  on  the  other  side  to  be  darker  than  the  path.  Two  RGB  threshold  values  were  chosen  from 
the  image  database  corresponding  to  these  conditions.  To  find  these  RGB  threshold  values,  first,  the  set  of  images 
corresponding  to  the  early  morning  and  evening  conditions  was  taken.  The  highest  of  the  RGB  pixel  values  in  the  trees 
on  the  side  that  is  darker  than  the  path  was  chosen  as  the  threshold  value.  Next,  the  sunny  day  condition  images  were 
taken  and  the  highest  of  the  RGB  pixel  values  in  the  trees  on  both  sides  of  the  path  was  taken  as  the  threshold  value. 
To  accommodate  these  conditions  automatically,  an  adaptive  thresholding  method  was  used  as  shown  in  Fig.  4. 

Next,  a  series  of  morphological  operations  were  performed  to  clearly  segment  the  path  and  a  line  was  fit  for  the 
boundaries,  using  the  least  squares  method.  The  error  was  calculated  as  the  distance  from  the  path  center.  Camera 
calibration  was  done  to  convert  pixel  distance  to  true  distance.  This  was  done  by  marking  known  distances  on  the 
ground  in  the  camera’s  field  of  view  and  the  number  of  pixels  corresponding  to  these  distances  was  counted.  To 
eliminate  fluctuation  in  path  tracking  due  to  natural  light  and  camera  noise,  a  time  based  outlier  elimination  routine 
was  added  to  the  error  calculation.  This  routine  assumes  that  the  error  data  in  a  short  interval  of  time  of  the  order  of 
1  s  is  normally  distributed.  When  a  new  error  data  is  found  to  lie  outside  two  standard  deviations  of  the  data  from 
the  previous  time  interval,  it  is  determined  as  noise  and  is  not  used  to  change  the  steering  for  the  present  time  step. 
If  however  successive  error  data  are  in  the  same  range,  then  they  are  taken  into  account  for  correcting  the  steering. 
An  example  of  grove  path  estimation  using  the  machine  vision  algorithm  is  shown  in  Fig.  5.  Note  the  path  boundary 
depicted  by  the  superimposed  straight  lines  along  the  tree  boundary.  For  navigation  experiments  on  the  test  path,  the 
color  threshold  for  the  machine  vision  algorithm  was  adopted  for  detecting  hay  bales  instead  of  trees. 

The  laser  radar  navigation  algorithm  employed  a  threshold  distance  based  detection  of  hay  bales.  The  flowchart 
of  the  algorithm  is  shown  in  Fig.  6.  The  plot  of  the  radial  distance  measured  by  the  ladar  for  different  angles,  when 
driving  through  the  test  path,  is  shown  in  Fig.  7.  The  discontinuities  in  the  plot  indicate  the  location  of  the  hay  bales. 
While  scanning  the  field  of  view  using  laser  radar,  if  such  discontinuities  are  found  in  the  data,  they  are  marked  as  hay 
bales.  The  path  center  was  determined  as  the  center  of  the  path,  between  the  hay  bales  on  either  side. 


Fig.  5.  Machine  vision  results  for  citrus  grove  alleyway:  (a)  raw  image;  (b)  tree  canopy  segmentation;  (c)  path  boundary. 
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Fig.  6.  Ladar  algorithm  flowchart. 


For  the  determination  of  the  vehicle’s  guidance  dynamics,  open  loop  frequency  response  tests  were  conducted  in 
an  open  field  scenario.  The  tractor  engine  was  set  at  2200 rpm  and  three  different  speeds  of  1.8  m/s  (slow),  3.1  m/s 
(medium)  and  4.4  m/s  (fast)  were  set  by  varying  the  gears.  As  the  tractor  was  run,  the  wheels  were  automatically 
turned  as  a  time  based  sinusoid.  The  tractor’s  position  was  recorded  in  real-time  using  the  DGPS  receiver.  The  tests 
were  conducted  in  an  open  field  without  any  trees  or  buildings  blocking  the  signal  to  the  DGPS  receiver.  The  receiver 
communicated  with  the  PC  via  RS  232  serial  communication.  The  steering  turn  frequency  was  performed  from  0.07 
to  0.3  Hz.  Frequencies  below  0.07  Hz  caused  the  tractor  to  veer  out  of  the  test  area  and  frequencies  above  0.3  Hz  were 
thought  to  be  detrimental  to  the  tractor. 

It  was  observed  that  the  vehicle’s  position  information  obtained  using  the  DGPS  receiver  showed  a  sinusoidal 
response  at  the  same  frequency  as  the  steering  angle.  The  lateral  displacement  of  the  vehicle  for  a  given  change  in 
steering  angle  was  recorded  as  the  gain,  where  the  lateral  displacement  is  the  displacement  of  the  vehicle  in  a  direction 
perpendicular  to  the  direction  of  initial  motion.  The  lateral  displacement  of  the  vehicle  was  calculated  from  the  latitude 
and  longitude  data  obtained  from  the  DGPS  receiver  using  Arc  View  3.3  (ESRI  Inc.,  Redlands,  CA).  The  accuracy  of 
Arc  View  3.3  was  verified  with  actual  manually  measured  distance  for  a  few  test  runs.  The  gain  was  plotted  against 
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Fig.  7.  Radial  distance  measured  by  the  laser  radar  in  the  hay  bale  path. 


Fig.  8.  Open  loop  theoretical  and  actual  frequency  response  of  the  sinusoidal  vehicle  dynamics  test  (phase  =  —180°). 


frequency  in  a  bode  plot  as  shown  in  Fig.  8.  The  response  was  observed  to  be  linear.  Visually  fit  lines  were  used  to 
model  this  response. 

The  sinusoidal  guidance  dynamics  data  was  used  to  calculate  the  transfer  function  between  the  steering  angle  and 
lateral  displacement  as 

lateral_displacement(s)  k 

steering_angle_change(s)  s2 

The  gain  k  was  speed  sensitive  and  was  determined  as  0.02  (1.8  m/s),  0.07  (3.1  m/s)  and  0.13  (4.4  m/s).  This  model 
agreed  with  the  model  reported  by  Stombaugh  et  al.  (1999). 


Error  in  Desired 


Fig.  9.  Simulated  vehicle  control  system  block  diagram. 
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Based  on  the  determined  theoretical  model,  a  PID  control  system  was  designed  (Fig.  9).  The  PID  gains  were  tuned 
by  simulation  in  Simulink,  using  the  Ziegler  Nichols  method  of  tuning  for  a  closed  loop  system.  Tuning  was  optimized 
for  zero  steady  state  error  and  low  overshoot.  Fig.  10  shows  the  1  m  step  response  when  simulated  for  the  3.3  m/s  speed. 

4.  Experimental  procedure 

Typical  citrus  grove  alleyways  are  practically  straight  rows.  It  was  felt  that  testing  the  vehicle  in  straight  and  curved 
paths  at  different  speeds  would  be  a  more  rigorous  test  of  the  guidance  system’s  performance  than  testing  in  straight 
grove  alleyways.  For  this  purpose,  a  flexible  wall  track  was  constructed  using  hay  bales  to  form  a  test  path.  The  hay 
bales  provide  a  physically  measurable  barrier,  which  aids  in  operating  the  ladar  guidance  system  and  color  contrast 
with  grass  which  is  useful  for  vision-based  guidance.  In  addition,  when  testing  a  large  tractor,  minor  losses  would  occur 
if  the  tractor  runs  over  a  barrier  of  hay  bales.  The  control  algorithms  used  with  machine  vision  or  ladar-based  guidance 
are  practically  identical  for  either  test  track.  The  only  modification  in  the  vision  algorithm  was  the  threshold  value  for 
image  segmentation.  These  threshold  values  were  determined  for  both  paths  using  techniques  described  earlier.  The  PID 
controller  and  the  encoder  performance  are  independent  of  the  physical  environment  around  the  vehicle.  The  hay  bale 
width  was  45  cm  and  the  boundary  height  was  90  cm  after  stacking  hay  bales  two  deep.  Two  types  of  path  were  used, 
a  straight  path  and  a  curved  path.  The  length  of  the  straight  path  was  22  m.  A  10  m  extension  was  added  to  the  straight 
path  to  form  the  curved  path.  The  radius  of  curvature  was  17  m  at  the  curve.  The  path  width  was  3.5  m  throughout  the 
length.  The  paths  used  in  this  study  to  evaluate  the  guidance  system  performance  are  shown  in  Fig.  1 1(a)  and  (b). 

Experiments  were  conducted  to  check  the  robustness  and  accuracy  of  the  vision  and  laser  radar  guidance  systems 
in  navigating  the  different  test  paths  at  different  speeds.  Experiments  were  conducted  for  three  different  speeds,  1.8, 
3.1  and  4.4  m/s.  These  speeds  corresponded  to  the  perceived  slow,  medium  and  fast  speeds  for  a  tractor  operating  in  a 
grove.  The  speeds  were  measured  using  an  ultrasonic  ground  speed  sensor. 


Fig.  11.  Guidance  system  test  path:  (a)  straight  and  (b)  curved. 


138 


V.  Subramanian  et  al.  /  Computers  and  Electronics  in  Agriculture  53  (2006)  130-143 


Fig.  12.  Vehicle  in  the  citrus  grove  alleyway. 


The  vehicle  was  manually  driven  to  the  starting  position,  resulting  in  different  initial  offset  and  heading  angles  for 
each  repetition.  Once  positioned,  the  vehicle  was  started  under  autonomous  control  and  allowed  to  navigate  down  the 
path.  Three  repetitions  were  performed  for  each  experiment.  A  rotating  blade  was  attached  to  the  draw  bar,  which 
marked  a  line  on  the  ground  as  the  vehicle  moved.  This  line  represented  the  path  center  traveled  by  the  tractor.  The 
errors  were  manually  measured  after  each  run  using  the  distance  from  the  marked  line  to  the  hay  bale  boundary.  The 
error  measurements  were  accurate  to  0.1  cm.  The  measurements  were  taken  at  regular  intervals  of  30  cm.  Due  to  the 
unpredictable  nature  of  the  starting  and  ending  data,  the  first  5%  and  last  5%  of  the  data  record  was  discarded  for 
statistical  analysis.  However,  the  plots  show  the  full  data  record,  which  illustrates  the  random  nature  of  the  vehicle 
starting  position. 

Data  points  were  taken  from  each  run  as  described  above  in  order  to  calculate  the  path  root  mean  square  error, 
standard  deviation,  maximum  error  and  average  error.  The  rms  error  equation  is  shown  in  Eq.  (2): 


rms  error  = 


N  2 


Mtr 


(2) 


where  e-t  is  the  path  error  and  A  is  the  total  number  of  data  points.  These  measures  were  used  to  compare  the  performance 
of  the  two  guidance  sensors,  the  two  different  path  configurations  and  the  three  speeds. 

After  achieving  satisfactory  performance  in  the  test  path,  trials  were  conducted  in  an  alleyway  of  the  citrus  grove. 
For  initial  grove  trials,  only  machine  vision  was  used  due  to  machine  vision  easily  adapting  to  grove  conditions.  Ladar, 
on  the  other  hand,  requires  that  additional  data  filters  be  designed  to  compensate  for  noise  due  to  random  orientation  of 
the  tree  leaves.  Future  development  is  planned  to  test  ladar  in  the  grove.  The  machine  vision  guidance  system  was  used 
to  guide  the  tractor  through  an  alleyway  of  a  citrus  grove  on  the  University  of  Florida  campus  as  shown  in  Fig.  12.  The 
average  path  width  was  4.6  m.  The  tractor  was  driven  to  the  grove  and  started  in  the  automatic  mode  at  the  beginning 


Table  1 

Performance  measures  of  the  vehicle’s  guidance  system  obtained  from  the  experiments  conducted  in  the  straight  test  path 


Sensor 

Speed  (m/s) 

Average  error  (cm) 

Maximum  error  (cm) 

Standard  deviation  (cm) 

rms  error  (cm) 

1.8 

2 

3.9 

0.8 

2.2 

Vision 

3.1 

2.8 

5.9 

1.2 

3.1 

4.4 

2.8 

6.1 

1.5 

3.2 

1.8 

1.6 

3.3 

0.7 

1.8 

Ladar 

3.1 

1.6 

3.9 

0.8 

1.8 

4.4 

3.1 

6.1 

1.6 

3.5 
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Table  2 

Performance  measures  of  the  vehicle’s  guidance  system  obtained  from  the  experiments  conducted  in  the  curved  test  path  at  3.1  m/s 


Sensor 

Path 

Average  error  (cm) 

Maximum  error  (cm) 

Standard  deviation  (cm) 

rms  error  (cm) 

Straight  section  (22  m) 

2.8 

5.1 

1.2 

3.1 

Vision 

Curved  section  (10  m) 

2.7 

3.9 

0.4 

2.7 

Overall  (32  m) 

2.8 

5.1 

0.9 

2.9 

Straight  section  (22  m) 

1.6 

3.9 

0.8 

1.8 

Ladar 

Curved  section  (10  m) 

4.1 

6.1 

2.9 

4.9 

Overall  (32  m) 

2.5 

6.1 

1.6 

2.9 

of  an  alleyway.  The  guidance  system  was  allowed  to  guide  the  tractor  through  the  entire  length  of  the  alleyway.  The 
performance  was  visually  observed. 

5.  Results  and  discussion 

Tests  were  conducted  on  each  of  the  sensor  and  track  conditions  to  observe  the  behavior  of  the  vehicle  under  various 
speeds.  During  each  trial  run,  the  path  position  and  error  were  recorded  manually  using  a  tape  measure.  The  rms  error, 


Fig.  13.  Performance  of  the  machine  vision  guidance  in  the  straight  path:  (a)  1.8  m/s;  (b)  3.1  m/s;  (c)  4.4  m/s. 
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average  error,  standard  deviation  of  the  error  and  the  maximum  error  were  calculated  for  each  experiment,  by  averaging 
the  data  over  the  three  repetitions  of  each  condition.  The  results  were  then  compared  for  the  different  conditions.  The 
path  error,  which  is  the  deviation  from  the  centerline  of  the  aisle  way,  was  plotted  over  the  length  of  the  test  path  for  the 
different  conditions.  Path  errors  were  adjusted  for  local  variation  in  hay  bales.  The  performance  measures  are  shown 
in  Table  1  for  straight  path  and  Table  2  for  curved  path.  It  should  be  noted  that  Table  2  data  is  broken  down  to  the 
straight,  curved  and  overall  path  errors. 

Fig.  13  shows  the  performance  of  the  machine  vision  guidance  system  at  the  three  different  test  speeds.  The  average 
error  was  less  than  3  cm  for  all  speeds  as  noted  in  Table  1.  The  performance  was  better  at  1.8  m/s  than  at  the  two 
higher  speeds.  As  shown  in  Fig.  13,  the  PID  control  is  attempting  to  drive  the  error  to  zero.  The  rms  error  is  3.2  cm  at 
the  fastest  speed  and  demonstrates  a  good  control  performance  visually  similar  to  human  driving.  The  machine  vision 
guidance  performed  well  in  the  straight  path.  The  performance  was  better  at  lower  speeds. 

Fig.  14  shows  the  performance  of  the  laser  radar  guidance  system  at  the  three  different  speeds.  Once  again  the 
PID  control  is  driving  the  error  around  zero.  The  average  error  at  the  lowest  speed  (1.6  cm)  is  almost  half  the 
error  at  the  highest  speed  (3.1cm).  Laser  radar  is  a  more  accurate  measuring  device  than  machine  vision  result¬ 
ing  in  lower  tracking  errors,  at  the  two  lower  speeds.  However  the  ladar  error  is  10.7%  higher  at  3.1cm  when 
traveling  at  4.4  m/s.  The  machine  vision  algorithm  was  run  at  20  Hz,  whereas  the  laser  radar  refresh  rate  was  only 
6  Hz  at  half  degree  resolution  when  using  a  standard  serial  port  communicating  at  115,200 baud.  The  slower  ladar 
refresh  rate  means  that,  less  current  data  is  available  for  the  laser  radar  guidance  system  than  the  machine  vision 


Fig.  14.  Performance  of  the  laser  radar  guidance  in  the  straight  path:  (a)  1.8  m/s;  (b)  3.1  m/s;  (c)  4.4  m/s. 
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(b)  Travel  Distance  (cm) 


Fig.  15.  Performance  in  the  curved  path  at  3.1  m/s:  (a)  machine  vision  guidance  and  (b)  laser  radar  guidance. 


guidance  system,  in  a  given  time  interval.  At  high  speeds  of  4.4  m/s,  this  factor  reduces  the  performance  of  the 
laser  radar  guidance  system,  whereas  the  machine  vision  guidance  system  is  not  severely  affected.  By  using  a  high 
speed  serial  communication  card  (500kbaud),  the  laser  radar  refresh  rate  can  be  increased  and  therefore  its  perfor¬ 
mance  at  speeds  4.4  m/s  and  higher,  can  be  improved.  None  the  less,  the  rms  error  was  only  3.5  cm  at  the  fastest 
speed. 

Fig.  15  shows  the  performance  of  the  vision  and  laser  radar  guidance  systems  at  a  speed  of  3.1  m/s  in  the  curved 
path.  The  performance  of  the  two  systems  was  visually  as  good  as  a  human  driving  the  tractor.  The  vision  guid¬ 
ance  had  an  average  error  over  the  entire  track  of  2.8  cm  whereas  the  laser  radar  guidance  system  had  an  average 
error  of  2.5  cm  as  shown  in  Table  2.  The  maximum  error,  rms  error  and  standard  deviation  of  machine  vision-based 
guidance,  when  run  at  3.1  m/s  in  curved  path  seems  to  have  done  better  than  in  the  straight  path.  It  is  to  be  noted 
that  the  curved  path  consists  of  a  straight  section  followed  by  a  curved  section.  The  camera’s  wide  field  of  view  is 
smaller  than  that  of  the  ladar  and  therefore  when  the  vehicle  gets  to  the  curved  section,  only  one  side  of  the  path 
boundary  is  visible  in  the  field  of  view.  The  machine  vision  tries  to  guides  the  vehicle  at  a  constant  distance  from 
the  visible  boundary.  This  can  be  observed  in  Fig.  15a  (vision  in  curved  path  at  3.1  m/s).  Since  the  curve  starts  at 
some  where  around  2100  cm  travel  distance,  Fig.  15a  shows  that  the  vision-based  PID  control  does  not  attempt  to 
drive  the  error  to  the  center  of  the  path  once  the  tractor  enters  the  curve.  However,  the  ladar-based  PID  control  in 
Fig.  15b  does  continue  to  cross  the  zero  error  path  center  in  the  curve  section.  The  maximum  error,  rms  error  and 
standard  deviation  of  error  in  this  curved  section  are  lower  than  that  for  the  straight  section.  When  averaged  over  the 
entire  path,  the  error  parameters  appear  lower  for  the  curved  path  than  for  the  straight  path.  The  laser  radar  guidance 
system’s  overall  accuracy  was  slightly  more  accurate  than  the  vision  guidance  when  considering  average  error.  As 
before,  it  performed  better  in  the  straight  path  section  of  the  path,  but  was  less  accurate  in  the  curve  than  the  vision 
system.  It  also  seems  to  exhibit  larger  swings  about  zero  error  than  the  vision  system  as  evident  by  the  larger  maximum 
error.  This  may  be  due  to  some  natural  dampening  that  occurs  in  the  less  accurate  vision  measurement  algorithm. 
This  can  be  attributed  to  the  fact  that  the  laser  radar  is  more  accurate  at  detecting  the  path  center  than  the  vision 
guidance. 

The  good  performance  in  the  hay  bale  track  encouraged  trials  in  the  citrus  grove  alleyway.  The  guidance  system 
successfully  guided  the  tractor  through  the  alleyway.  Initial  trials  using  machine  vision  for  guidance  showed  a  lot  of 
promise.  The  machine  vision  algorithm  clearly  segmented  the  path  to  be  traversed.  However,  the  non-uniformity  of 


142 


V.  Subramanian  et  al.  /  Computers  and  Electronics  in  Agriculture  53  (2006)  130-143 


the  canopy  surface  caused  the  controller  to  over  compensate  for  the  undulation  in  canopy  surface  resulting  in  a  more 
zigzag  transit  of  the  alleyway.  Control  enhancements  are  planned  to  further  improve  the  guidance  system  performance 
in  the  grove. 

6.  Conclusion 

Machine  vision  and  laser  radar  based  guidance  systems  were  developed  to  navigate  a  tractor  through  the  alleyway  of 
a  citrus  grove.  A  PID  controller  was  developed  and  tested  to  control  the  tractor  using  the  information  from  the  machine 
vision  system  and  laser  radar.  A  flexible  curvature  track  was  built  using  hay  bales  as  boundaries  and  the  vehicle  was 
tested  under  straight  path  and  curved  path  configurations.  Tests  were  conducted  at  three  different  speeds  of  1.8,  3.1 
and  4.4  m/s.  Path  tracking  comparisons  were  made  between  the  vision-based  control  and  a  ladar-based  control.  The 
traversed  path  was  marked  on  the  ground  and  measured  manually. 

Error  comparisons  were  made  using  rms  error  and  mean  instantaneous  error.  It  was  found  that  the  ladar-based 
guidance  was  the  better  guidance  sensor  for  straight  and  curved  paths  at  speeds  of  up  to  3. 1  m/s.  Communication  speed 
between  the  laser  radar  and  the  computer  was  a  limiting  factor  for  higher  speeds.  This  problem  can  be  solved  by  a  faster 
communication  speed  using  a  high  speed  serial  communication  card  in  the  PC.  This  card  was  not  available  during  this 
research.  Machine  vision-based  guidance  showed  acceptable  performance  at  all  speeds  and  conditions.  The  average 
errors  were  below  3  cm  in  most  cases.  The  maximum  error  was  not  more  than  6  cm  in  any  test  run.  These  experiments 
demonstrated  the  accuracy  of  the  guidance  system  under  test  path  conditions  and  successful  guidance  of  the  tractor  in 
a  citrus  grove  alleyway.  However,  additional  testing  is  needed  to  improve  the  performance  in  the  citrus  grove.  It  was 
proposed  that  a  control  scheme,  which  used  both  machine  vision  and  laser  radar,  may  provide  a  more  robust  guidance, 
as  well  as  provide  obstacle  detection  capability. 
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